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PREFACE 


CUSTOMER  SATISFACTION  REPORTING 
CONDUCTED  BY  THE  DEFENSE  TECHNICAL  INFORMATION  CENTER 

The  Defense  Technical  Information  Center  (DTIC)  has  surveyed  its 
registered  customers  since  1999.  There  are  two  surveys 
conducted : 

Top  200  Users.  Defined  in  terms  of  the  dollar  value  of  billing, 
number  of  documents  ordered,  and  downloads  from  Secure  STINET. 

All  200  users  are  surveyed.  Surveys  were  conducted  in  FY  2000, 

FY  2001  and  will  be  conducted  in  FY  2002. 

Customer  Satisfaction  Survey.  A  random  sample  of  all  users  not 
included  in  the  Top  200  Survey.  Surveys  were  conducted  in  FY 
1999,  FY  2001  and  FY  2002. 

Selected  data  elements  were  combined  into  the  2001  Annual  Survey 
Composite  Data  Report.  A  similar  combined  report  is  planned  for 
FY  2002. 

The  DTIC  Performance  Report  to  the  Defense  Information  Systems 
Agency  (DISA)  contains  a  report  on  overall  customer  satisfaction. 
This  is  derived  from  the  Annual  Survey  Composite  Data  Report  and 
reflects  data  from  both  the  Top  200  Users  Survey  and  the  Customer 
Satisfaction  Survey. 


Customer  Satisfaction  Survey  Results: 


How  Do  We  Measure  Up? 


DTIC's  general  population  customer  satisfaction  scores 
continue  to  rise  and  surpass  the  Federal  Government 
American  Customer  Satisfaction  Index  (ACSI) 
baseline/benchmark  score .  See  chart  below  for 
illustration : 


Customer  Service 
Elements 

FY  99 
Customer 
Satisfaction 
Survey 

FY  2001 
Customer 
Satisfaction 
Survey 

FY2002 

Customer 

Satisfaction 

Survey 

DTIC  Composite  Scores 

74% 

76% 

81% 

ACSI  Federal  Government 
Benchmark 

68.6% 

68.6% 

71.1% 

DTIC  Excels  By 

+5.4 

+7.4 

+9.9 

Note:  Data  relevant  to  this  chart  is  detailed  in  Part  II 
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FY  2002  Customer  Satisfaction  Survey  Report 


Part  I :  Introduction 

Purpose:  In  April  2002,  the  Defense  Technical  Information  Center 
(DTIC)  conducted  its  annual  customer  satisfaction  survey  to  help 
us  better  understand  how  to  serve  our  user  community.  The 
overall  objective  of  this  survey  was  to  receive  insight  in 
measuring  the  effectiveness  of  DTIC's  products  and  customer 
services.  This  report  summarizes  the  results  of  the  FY  2002 
Customer  Satisfaction  Survey  (FY  2002  CS  Survey) .  It  also 
compares  results  from  the  FY  2001  and  FY  1999  Customer 
Satisfaction  Survey  findings.  In  addition,  the  survey  queried 
customers  in  six  areas: 

■  Customer  Service  Experiences 

■  Global  Customer  Service  Performance 

■  DTIC  Products  and  Services 

■  DTIC  Online  Services 

■  User  Demographics 

■  Communication/Access  and  Information  Requirements 

Questions  on  the  survey  were  geared  toward  customer  service 
factors:  accessibility,  accuracy,  courtesy,  helpfulness, 
knowledge,  professionalism,  responsiveness,  and  speed  of  service. 

Other  questions  targeted  aspecsa  bet.i.e.,he  survey  queried  T jvolvnf orm, -12aact 
omers  i!8  areas: 


The  survey  population  universe  for  this  effort  was  4,072.  A 
total  of  3, 083  users  were  randomly  selected  to  participate  in  the 
survey.  After  an  intensive  email  and  call  effort  lasting  three 
and  a  half  weeks,  463  or  15  percent  of  the  sample  population 
responded  and  were  qualified  as  "valid"  participants.  With  a 
universe  of  5,000  users,  an  expected  return  rate  of  15  percent, 
and  a  confidence  level  of  95  percent  with  a  5  percent  error 
tolerance,  the  sample  size  would  have  to  be  at  least  2,380  to  be 
statistically  valid.  Since  the  sample  surveyed  was  3, 083  users, 
the  survey  results  are  considered  statistically  valid  based  on 
the  above  criteria.  An  average  of  858  of  the  2,961  email 
notifications  sent  out  were  returned  as  non-deliverable.  The 
following  are  possible  reasons: 

•  Email  address  on  file  not  valid 

•  User  no  longer  at  email  address 

•  Increased  security  due  to  911  shut  down  agency  email  systems 

•  Server  problems  for  recipient  or  originator 

Important  Note:  When  considering  the  results  of  Part  1  of  the 
survey  process,  it  is  important  to  recognize  that  25%  of  the 
universe  was  not  surveyed,  and  that  of  the  3,083  surveyed,  15% 
responded.  We  believe  that  those  who  were  surveyed  but  did  not 
respond  are  more  likely  not  to  consider  themselves  DTIC  users,  or 
not  to  perceive  themselves  as  having  a  sufficient  stake  in  DTIC 
to  take  the  time  to  respond.  Analysis  of  the  survey  results 
will  always  need  to  be  tempered  by  considerations  of  who  did  and 
who  did  not  respond,  and  to  what  extent  those  who  responded  have 
the  same  characteristics  or  views  as  those  who  did  not  respond. 

Overall  Findings 


User  Demographics 

♦  The  majority  of  respondents  overall  classified  their  job 
position  as  Librarian  who  primarily  acts  as  information 
provider . 

♦  The  majority  of  Department  of  Defense  (DOD)  respondents  in 
particular  overwhelmingly  reported  themselves  as  end  users. 

♦  The  majority  of  users  responding  came  from  three  user 

groups:  Librarians,  Engineers  and  Researcher/Analysts . 

♦  More  than  a  third  of  DTIC  users  reported  being  registered  for 
six  years  or  more. 

♦  Less  than  one  fifth  of  users  reported  being  registered  with 
DTIC  for  less  than  one  year. 
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Communication/Access 


♦  Users  find  that  the  majority  of  their  information  needs  are 
met  by  searching  DTIC's  collections. 

♦  The  majority  of  respondents  find  the  Internet/Online  services 
as  the  most  often  used  communication  channel  to  contact  DTIC. 

♦  The  vast  majority  of  customers  reported  that  it  is  very 
important  to  extremely  important  to  speak  with  a  "live  person" 
when  calling  DTIC. 

♦  The  majority  of  users  prefer  email  as  the  primary  means  of 
receiving  informational  updates. 

Marketing  Communication 

♦  Users  reported  that  they  are  satisfied  that  DTIC  does  an 
adequate  job  in  communicating  (transmitting/mailing) 
information  about  new  changes  in  and/or  enhanced  products  and 
services  offerings,  training  opportunities,  etc. 

♦  A  third  of  all  respondents  reported  reading  the  DTIC  DIGEST. 

♦  Nearly  half  of  all  respondents  prefer  the  hardcopy  version  of 
the  DTIC  DIGEST  to  the  electronic  version. 

Customer  Service  Issues 

♦  81  percent  of  surveyed  respondents  reported  that  they  were 
satisfied  with  the  level  of  DTIC's  customer  care. 

♦  The  customer  satisfaction  service  ratings  have  shown  an  upward 
trend  in  comparison  to  previous  years. 


Online  Services  (Public  STINET,  Secure  STINET,  Web-Enabled 
DROLS,  Classified  DROLS) 

♦  Public  STINET  is  still  the  most  popular  online  service. 

♦  Overall  online  service  quality  performance  rating  was  low. 

♦  Customer  support  for  online  services  rating  decreased  10 
percentage  points  from  FY  2001. 

♦  Overall,  Librarians  tend  to  rate  online  services  the  highest 
of  DTIC's  top  3  user  groups. 
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Other  DTIC  Products  and  Services  -  (as  differentiated  from 

online  Services ) 

♦  Overall  product  satisfaction  has  remained  the  same  for  the 
past  3  years. 

♦  Product  usage  increased  50  percent. 

♦  Customer  satisfaction  with  DTIC' s  delivery  of  documents  has 
steadily  increased  over  the  last  3  years . 

General  Findings 

♦  The  vast  majority  of  users  reported  that  DTIC  is  very 
important  to  the  accomplishments  of  their  business  objectives. 

♦  A  significant  majority  of  users  reported  that  DTIC  has  a  high 
to  very  high  impact  in  supporting  their  overall  mission. 

♦  The  vast  majority  of  users  would  recommend  DTIC  to  a 
colleague . 
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Part  II :  Global  Customer  Service 
Performance  Rating  Review 


Overall  Customer  Satisfaction  Performance  Ratings:  With 
regard  to  the  FY  2002  CS  Survey,  an  impressive  number  of 
respondents  (81  percent)  reported  that  they  were  satisfied  with 
the  level  of  DTIC's  customer  care.  The  FY  2002  composite 
customer  service  score  is  up  by  5  points,  from  76  percent,  since 
last  year.  Aspects  of  the  customer  satisfaction  service  ratings 
have  shown  an  upward  trend  in  comparison  to  previous  years.  This 
can  be  illustrated  in  the  graph  (Fig  2.1)  below: 


Comparative  Analysis  of  Customer 
Satisfaction  (Customer  Care)  Performance  Rating 
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Fig  2.1 


In  order  to  produce  the  customer  service  composite  score, 
respondents  were  asked  to  rate  their  level  of  satisfaction  with 
DTIC's  customer  care  in  eight  (8)  distinct  quality  areas.  Those 
critical  care  elements  and  response  data  are  displayed  below  in 
Fig  2.2.  With  regard  to  aspects  of  this  year's  customer  care 
quality  factors,  all  had  shown  improvement.  The  most  improvement 
has  been  DTIC's  ability  to  effectively  respond  to  user  inquiries 
and  problems  in  a  timely  manner,  both  up  6  points. 
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DTIC's  Overall  Customer  Care 
Quality  Factors 

FY  2001 
Customer 
Satisfaction 
Survey 

FY  2002 
Customer 
Satisfaction 
Survey 

Performance 

Gap 

Accessibility  (Easy  to  access  staff) 

72% 

77% 

+5 

Accuracy  (Provided  correct  info) 

76% 

80% 

+4 

Courtesy 

81% 

85% 

+4 

Helpfulness 

79% 

83% 

+4 

Knowledge 

77% 

83% 

+6 

Professionalism 

81% 

85% 

+4 

Responsiveness(Timely  response) 

72% 

78% 

+6 

Speed  of  Service 

71% 

77% 

+6 

Overall  Rating 

76% 

81% 

Note:  FY  99  CS  survey  had  similar  quality  factor  descriptors,  but  varied  in  some  instances 
in  comparison  to  FY  2001  and  FY  2002  CS  surveys  (i.e.,  follow-up  call,  problem  resolved  to 
your  satisfaction  and  communication  skills).  Fig  2.  2 


Comparative  Benchmarking-Comparison  to  American  Customer 

Satisfaction  Index  (ACSI) 

DTIC's  Customer  Service  Results: 

■  Results  obtained  from  the  2001  Federal  Government  American 
Customer  Satisfaction  Index  (ACSI)  have  been  measured  against 
53  different  customer  groups  served  by  39  federal  government 
agencies . 

■  The  2001  December  aggregated  ACSI  score  for  the  Federal 
Government  is  71.1  percent,  up  3.5  points  from  68.6  percent 
since  last  year. 

■  DTIC's  FY  2002  Overall  Customer  Satisfaction  composite  score 
is  81  percent. 

■  When  measured  in  terms  of  the  2001  ACSI,  it  is  above  the 
established  federal  baseline/benchmark  score  by  9.9  points. 
Note:  Index  scores  detailed  in  Part  II  of  this  summary  do  not 
include  Top  200  users.  FY  2002  Top  200  users'  scores  will  be 
integrated  once  the  survey  is  completed.  This  will  result  in 
the  established  FY  2002  DTIC  baseline  customer  satisfaction 
performance  rating. 


Customer  Service 
Elements 

FY  99 
Customer 
Satisfaction 
Survey 

FY  2001 
Customer 
Satisfaction 
Survey 

FY2002 

Customer 

Satisfaction 

Survey 

DTIC  Composite  Scores 

74% 

76% 

81% 

ACSI  Federal  Government 
Benchmark 

68.6 

68.6% 

71.1% 

DTIC  Excels  by 

+5.4 

+7.4 

+9.9 

Fig  2  .3 
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Importance  of  DTIC  to  Accomplishment  of  Business  Objectives 
and  Mission  Support: 

The  vast  majority  of  users  reported  that  DTIC  is  important  to  the 
accomplishment  of  their  business  objectives  (86  percent)  and 
mission  support  (82  percent) .  In  addition,  over  half  of  the 
respondents  see  DTIC' s  role  as  "very  to  extremely  important"  in 
their  business  accomplishment. 


Fig  2.4 


Note:  This  is  the  first  year  we  have  directly  asked  DTIC's  general  users  questions  on  business 
objectives  and  mission  support;  therefore,  no  comparable  data  are  available.  However,  these 
questions  have  been  asked  of  our  Top  200  users  in  previous  surveys. 


Fig  2.5 
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Recommending  DTIC  Product (s)  and/or  Service (s)  to 
Colleagues : 

Overall,  91  percent  of  respondents  would  recommend  DTIC  to 
colleagues.  FY  2002  CS  survey  data  on  this  issue  show  a  7  point 
increase  over  FY  2001.  These  high  numbers  are  also  supported  by 
the  numerous  written  comments  from  respondents  indicating  their 
need  and  support  for  DTIC's  type  of  services.  See  Part  VII  for  a 
sampling  of  supportive  user  comments. 
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Fig  2.6 


Delivery  Processes : 

Users  responding  to  the  FY  2002  CS  Survey  reported  that  DTIC  does 
a  good  job  at  meeting  anticipated  delivery  schedules.  It  is 
interesting  to  note  that  22  percent  of  users  indicated  that  they 
download  documents  themselves.  Document  download  data  are  not 
available  for  FY  2001  and  1999.  It  is  also  interesting  to  note 
that  a  large  percentage  of  Engineers  and  Researcher/Analysts  (32 
percent  and  38  percent  respectively)  reported  downloading 
documents  themselves,  while  only  8  percent  of  Librarians  reported 
downloading  documents  themselves. 


Fig  2.7 


Customer  Satisfaction  with 
Meeting  Delivery  Schedules 

FY  1999 
Customer 
Satisfaction 
Survey 

FY  2001 
Customer 
Satisfaction 
Survey 

FY  2002 
Customer 
Satisfaction 
Survey 

Every  time  to  most  of  the  time 

55% 

65% 

71% 

About  Half  the  time 

30% 

26% 

5% 

Never  to  less  than  half 

15% 

9% 

2% 

*Download  documents  themselves 

N/A* 

N/A* 

22% 

*Data  not  gathered  in  previous  survey 

Fig  2.  8 
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DTIC  Online  Service  Overall  Quality  and  Performance : 


Users  participating  in  the  FY  2002  CS  Survey  reported  the 
following:  54  percent  of  users  rated  "Online  Service  Quality"  as 
"Very  Good"  to  "Excellent,"  21  percent  as  "Good,"  and  25  percent 
as  "Fair"  to  "Poor."  More  important  are  the  ratings  for 
individual  online  services  and  individual  quality  factors  which 
are  detailed  below.  Ratings  for  Classified  DROLS  were  low,  which 
brought  the  averages  for  all  online  services  down.  Also 
noteworthy  are  the  ratings  for  customer  support  quality  factor 
for  online  services.  Of  the  six  factors  targeted,  customer 
support  was  rated  the  lowest  at  53  percent.  These  scores  reflect 
an  overall  customer  satisfaction  rating  for  online  services  of  58 
percent  which  is  down  10  points  from  last  year. 

For  the  FY  2001  CS  Survey,  users  reported  as  follows:  60  percent 
of  users  rated  "Online  Service  Quality"  as  "Very  Good"  to 
"Excellent, "  32  percent  as  "Good, "  and  8  percent  as  "Fair"  to 
"Poor."  This  equates  to  an  overall  customer  satisfaction  rating 
for  online  services  of  60  percent,  combining  the  results  for 
"Excellent"  and  "Very  Good" . 


Comparative  Analysis  of  DTIC' s  Online  Services  Performance  Rating 

1  FY  2001  Customer  Satisfaction 

1  Survey 

Favorable% 

Neutral% 

Unfavorable% 

Accessibility  (Access  to  site) 

71% 

24% 

5% 

Ease  of  Use  (Navigating  the  site) 

55% 

34% 

11% 

Content 

60% 

33% 

7% 

Organization 

56% 

34% 

11% 

Usability  (Information  useful) 

59% 

35% 

7% 

Customer  Support 

61% 

33% 

7% 

Overall  Rating 

60% 

32% 

8% 

1  FY  2002  Customer  Satisfaction 

1  Survey 

Favorable% 

Neutral% 

Unfavorable% 

Accessibility  (Access  to  site) 

57% 

19% 

24% 

Ease  of  Use  (Navigating  the  site) 

52% 

25% 

24% 

Content 

56% 

21% 

23% 

Organization 

51% 

25% 

24% 

Usability  (Information  useful) 

55% 

21% 

24% 

Customer  Support 

50% 

16% 

34% 

Overall  Rating 

54% 

21% 

25% 

Fig  3.3 
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Users  responding  to  the  FY  2002  CS  Survey  were  asked  to  rate  the 
level  of  online  service  using  several  quality  factors  below.  The 
following  identifies  the  online  service  quality  rating  segmented 
by  the  top  three  user  groups,  Librarians,  Engineers,  and 
Researcher/Analysts : 


Accessibility  Overall  Ratings : 

•  57  percent  of  Users  rated  accessibility  as  "Very  Good"  to 
"Excellent" 

•  19  percent  as  "Good" 

•  24  percent  as  "Fair"  to  "Poor" 


Analysis  of  DTIC's  Online  Services  Performance  Rating  - 
Accessibility  (Access  to  site) 


FY  2002  Customer  Satisfaction 
Survey 

Favorable  % 

Neutral  % 

Unfavorable% 

Overall 

57% 

19% 

24% 

Librarian 

69% 

23% 

9% 

Engineer 

42% 

17% 

42% 

Researcher/Analyst 

55% 

19% 

26% 

Fig  3.4 


Ease  of  Use  (Navigating  the  site)  Overall  Ratings: 

•  52  percent  of  users  rated  navigation  as  "Very  Good"  to 
"Excellent" 

•  25  percent  as  "Good" 

•  24  percent  as  "Fair" 


Analysis  of  DTIC's  Online  Services  Performance  Rating  -  Ease  of 

Use  (Navigating  the  Site) 


FY  2002  Customer  Satisfaction 
Survey 

Favorable% 

Neutral% 

Unfavorable% 

Overall 

52% 

25% 

24% 

Librarian 

60% 

30% 

10% 

Engineer 

38% 

20% 

42% 

Researcher/Analyst 

53% 

19% 

28 

Fig  3.5 
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Content  Overall  Rating: 

•  56  percent  of  users  rated  content  as  "Very  Good"  to 
"Excellent" 

•  21  percent  as  "Good" 

•  23  percent  as  "Fair" 


Analysis  of  DTIC's  Online  Services  Performance  Rating  -  Content 


FY  2002  Customer  Satisfaction 
Survey 

Favorable% 

Neutral% 

Unfavorable% 

Overall 

56% 

21% 

23% 

Librarian 

65% 

26% 

9% 

Engineer 

41% 

16% 

43% 

Researcher/Analyst 

52% 

21% 

28% 

Fig  3.6 


Organization  Overall  Rating: 

•  51  percent  of  users  rated  organization  as  "Very  Good"  to 
"Excellent" 

•  25  percent  as  "Good" 

•  24  percent  as  "Fair" 


Analysis  of  DTIC's  Online  Services  Performance  Rating  - 

Organization 


FY  2002  Customer  Satisfaction 
Survey 

Favorable% 

Neutral% 

Unfavorable 

Overall 

51% 

25% 

24% 

Librarian 

57% 

33% 

11% 

Engineer 

37% 

21% 

43% 

Researcher/Analyst 

51% 

20% 

29% 

Fig  3.7 


Usability  Overall  Rating: 

•  55  percent  of  users  rated  usability  as  "Very  Good"  to 
"Excellent" 

•  21  percent  as  "Good" 

•  24  percent  as  "Fair 


| 

Analysis  of  DTIC' s  Online  Services  Performance  Rating  ■ 

-  Usability 

1 

FY  2002  Customer  Satisfaction 
Survey 

Favorable% 

Neutral% 

Unfavorable% 

Overall 

55% 

21% 

24% 

Librarian 

68% 

23% 

9% 

Engineer 

38% 

20% 

43% 

Researcher/Analyst 

58% 

13% 

29% 

Fig  3.8 


13 


Customer  Support  Overall  Rating: 

•  50  percent  of  users  rated  customer  support  as  "Very  Good"  to 
"Excellent" 

•  16  percent  as  "Good" 

•  34  percent  as  "Fair" 


Analysis  of  DTIC' s  Online  Services  Performance  Rating  -  Customer 

Support 


FY  2002  Customer  Satisfaction 
Survey 

Favorable% 

Neutral% 

Unfavorable% 

Overall 

50% 

16% 

34% 

Librarian 

58% 

20% 

21% 

Engineer 

37% 

10% 

53% 

Researcher/Analyst 

50% 

12% 

38% 

Fig  3.9 


Comparative  Analysis  of  DTIC's  Online  Services  Performance  Rating 


FY  2001  Customer  Satisfaction 

Favorable% 

Neutral% 

Unfavorable% 

Survey 

Accessibility  (Access  to  site) 

71% 

24% 

5% 

Ease  of  Use  (Navigating  the  site) 

55% 

34% 

11% 

Content 

60% 

33% 

7% 

Organization 

56% 

34% 

11% 

Usability  (Information  useful) 

59% 

35% 

7% 

Customer  Support 

61% 

33% 

7% 

Overall  Rating 

60% 

32% 

8% 

FY  2002  Customer  Satisfaction 

Favorable% 

Neutral% 

Unfavorable% 

Survey 

Accessibility  (Access  to  site) 

57% 

19% 

24% 

Ease  of  Use  (Navigating  the  site) 

52% 

25% 

24% 

Content 

56% 

21% 

23% 

Organization 

51% 

25% 

24% 

Usability  (Information  useful) 

55% 

21% 

24% 

Customer  Support 

50% 

16% 

34% 

Overall  Rating 

54% 

21% 

25% 

Fig  3.10 
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Performance  of  Individual  Services : 


Figures  3.11  to  3.14  depict  the  overall  rating  of  various  quality 
factors  for  each  online  service  for  the  FY  2002  CS  Survey. 

Public  STINET  Quality  Performance  Rating 


0  10  20  30  40  50  60  70  80  90  100 


(%) 


Fig  3.11 


Secure  STINET  Quality  Performance  Rating 

Accessibility 

165.8 

Ease  of  Use 

163.6 

Content 

165 

Organization 

161 .8 

Usability 

165.2 

Customer  Support 

156.8 

Overall  Average 

1 63.03 

10  20  30  40  50  60  70  80  90  1( 

)0 

(%) 

Fig  3.12 
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WED  Quality  Performance  Rating 

Accessibility 

1  65.2 

Ease  of  Use 

1  63.2 

Content 

1  64.6 

Organization 

162.2 

Usability 

163.6 

Customer  Support 

157.2 

Overall  Average 

1  62.67 

( 

10  20  30  40  50  60  70  80  90  1( 

(%) 

)0 

Fig  3.13 


Classified  DROLS  Quality  Performance  Rating 


0  10  20  30  40  50  60  70  80  90  100 

(%) 


Fig  3.14 
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Part  IV:  DTIC ' s  Other  Products  and  Services  Usage  Profile 


The  survey  results  show  that  64%  of  respondents  use  the  current 
awareness  product,  Research  Summaries  (RS) .  It  may  be,  however, 
that  respondents  did  not  distinguish  the  RS  database  from  the  RS 
current  awareness  service,  as  there  are  more  who  respond  as  users 
than  are  actually  registered  for  the  current  awareness  product. 

It  is  unclear  how  the  distortion  created  from  this  survey 
question  may  have  affected  other  aspects  of  this  data. 
Subscription  records  for  other  current  awareness  services  show  a 
reasonable  relationship  between  subscriptions  and  survey 
responses  for  the  other  products. 


Products  and  Services  Usage  Profile 


Service/Usage 

FY  2001 
Customer 
Satisfaction 
Survey 

FY  2002 
Customer 
Satisfaction 
Survey 

%  Point  Increase 

ADD 

2.3% 

12.7% 

+10.4 

CAB 

7.2% 

19.4% 

+12.2 

ECAB 

16.1% 

24.7% 

+8.6 

Research  Summaries 

29.7% 

63.8% 

+34.1 

Automatic  Magnetic  Tape 

Dist. 

0.0% 

2.0% 

+2.0 

TR  Database  CD-ROM 

9.6% 

30.4% 

+20.8 

Retrospective  Bibliography 
Magnetic  Tape 

1 .3% 

1 .4% 

+0.1 

The  DTIC  Review 

24.4% 

40.6% 

+16.2 

TRAIL 

Not  reported 

19.4 

N/A 

Fig  4.1 
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Percentage 


For  those  users  responding  to  the  survey,  detailed  products  and 
services  usage  is  provided  as  follows  including  a  breakdown  for 
the  top  3  job  positions: 


FY  2002  Comparison  of  Product  & 
Service  Usage  for 
Top  3  User  Groups 


u 

TRAIL 

DTIC 

Review 

RBMT 

TR-CD 

ROM 

AMTD 

RS 

ECAB 

CAB 

ADD 

■  Librarians 

25 

42 

3 

44 

0 

50 

38 

38 

17 

D  Enaineers 

17 

38 

0 

22 

0 

59 

4 

40 

0 

■  Researcher/Analvsts 

14 

37 

0 

33 

4 

71 

17 

18 

17 

Fig  4.2 


Overall  Product/Service  Quality  and  Performance: 

For  the  FY  2002  users  responding,  the  composite  products  and 
services  overall  quality  satisfaction  profile  showed  70  percent 
rating  "Very  Good"  to  "Excellent,"  27  percent  as  "Good,"  and  3 
percent  as  "Fair"  to  "Poor."  Ratings  were  similar  in  FY  2001 
showing  70  percent  as  "Very  Good"  to  "Excellent,"  24  percent  as 
good,  and  7  percent  as  "Fair"  to  "Poor."  Although  data  is 
limited  for  FY  1999,  overall  customer  satisfaction  with  DTIC 
products  remained  basically  the  same  at  70  percent  of  users 
responding  favorably,  27  percent  of  users  who  were  neutral  and 
only  4  percent  of  users  responding  negatively. 
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Comparative  Analysis  of  Overall 
Product/Service  Quality 


o- 

FY  99  Customer 
Satisfaction  Survey% 

FY  2001  Customer 
Satisfaction  Survey  % 

FY2002  Customer 
Satisfaction  Survey  % 

■  Very  Good  to  Excellent 

70 

70 

70 

□  Good 

25 

23 

27 

■  Fair  to  Poor 

5 

7 

3 

Fig  4.3 


Satisfaction  with  Individual  Products  and  Services : 


Individually,  in  FY  2002  the  products  and  services  receiving  the 
highest  marks  for  overall  quality  satisfaction  were  ECAB  (40 
percent).  Research  Summaries  (33  percent)  and  TR  Database  on  CD 
ROM  (32  percent) .  Individual  products  and  services  ratings  were 
not  available  for  FY  2001  and  1999.  Overall,  Librarians  rated  the 
individual  products  and  services  the  highest  while  Engineers  were 
the  most  negative  in  their  ratings.  Detailed  information  for 
several  DTIC  products  and  services  is  provided  below. 

Automatic  Document  Distribution  (ADD) : 

•  72  percent  of  users  rated  ADD  as  "Very  Good"  to  "Excellent" 

•  28  percent  as  "Good," 

•  0  percent  as  "Fair" 


Analysis  of  Products  &  Services  Performance  Rating  -  ADD 


FY  2002  Customer  Satisfaction 
Survey 

Favorable% 

Neutral% 

Unfavorable% 

Overall 

72% 

28% 

0% 

Librarian 

80% 

20% 

0% 

Engineer 

0% 

0% 

0% 

Researcher/Analyst 

29% 

71% 

0% 

Fig  4.4 
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Current  Awareness  Bibliography  (CAB) : 

•  69  percent  of  users  rated  CAB  as  "Very  Good"  to  "Excellent" 

•  29  percent  as  "Good" 

•  2  percent  as  "Fair" 


Analysis  of  Products  &  Services  Performance  Rating  -  CAB 


FY  2002  Customer  Satisfaction 
Survey 

Favorable% 

Neutral% 

Unfavorable% 

Overall 

69% 

29% 

2% 

Librarian 

65% 

35% 

0% 

Engineer 

1 00% 

0% 

0% 

Researcher/Analyst 

50% 

50% 

0% 

Fig  4.5 


Electronic  Current  Awareness  Bibliography  (ECAB) : 

•  71  percent  of  users  rated  ECAB  as  "Very  Good"  to  "Excellent" 

•  27  percent  as  "Good" 

•  2  percent  as  "Fair" 


Analysis  of  Products  &  Services  Performance  Rating  -  ECAB 


FY  2002  Customer  Satisfaction 
Survey 

Favorable% 

Neutral% 

Unfavorable% 

Overall 

71% 

27% 

2% 

Librarian 

63% 

37% 

0% 

Engineer 

1 00% 

0% 

0% 

Researcher/Analyst 

67% 

33% 

0% 

Fig  4.6 


Research  Summaries  (RS) : 

•  78  percent  of  users  rated  RS  as  "Very  Good"  to  "Excellent" 

•  18  percent  as  "Good" 

•  4  percent  as  "Fair" 
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Analysis  of  Products  &  Services  Performance  Rating  -  Research 

Summaries 


FY  2002  Customer  Satisfaction 
Survey 

Favorable% 

Neutral% 

Unfavorable% 

Overall 

78% 

18% 

4% 

Librarian 

75% 

25% 

0% 

Engineer 

84% 

4% 

12% 

Researcher/Analyst 

67% 

29% 

4% 

Fig  4.7 


Technical  Report  (TR)  Database  CD-ROM: 

•  79  percent  of  users  rated  TR  Database  on  CD-ROM  as  "Very  Good" 
to  "Excellent" 

•  15  percent  as  "Good" 

•  7  percent  as  "Fair" 


Analysis  of  Products  &  Services  Performance  Rating  -  TR  Database 

CD-ROM 


FY  2002  Customer  Satisfaction 
Survey 

Favorable% 

Neutral% 

Unfavorable% 

Overall 

79% 

15% 

7% 

Librarian 

80% 

12% 

8% 

Engineer 

80% 

0% 

20% 

Researcher/Analyst 

64% 

36% 

0% 

Fig  4.8 


The  DTIC  Review: 

•  78  percent  of  users  rated  the  DTIC  Review  as  "Very  Good"  to 
"Excellent" 

•  20  percent  as  "Good" 

•  2  percent  as  "Fair" 


Analysis  of  Products  &  Services  Performance  Rating  -  The  DTIC 

Review 


FY  2002  Customer  Satisfaction 
Survey 

Favorable% 

Neutral% 

Unfavorable% 

Overall 

78% 

20% 

2% 

Librarian 

81% 

19% 

0% 

Engineer 

79% 

14% 

7% 

Researcher/Analyst 

69% 

23% 

8% 

Fig  4.9 
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Technical  Report  Automated  Information  List  (TRAIL) : 

•  61  percent  of  users  rated  TRAIL  as  "Very  Good"  to  "Excellent" 

•  33  percent  as  "Good" 

•  7  percent  as  "Fair" 


Analysis  of  Products  &  Services  Performance  Rating  -  TRAIL 


FY  2002  Customer  Satisfaction 
Survey 

Favorable% 

Neutral% 

Unfavorable% 

Overall 

61% 

33% 

7% 

Librarian 

39% 

54% 

8% 

Engineer 

75% 

25% 

0% 

Researcher/Analyst 

43% 

43% 

14% 

Fig  4.10 
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PART  V:  User  Demographics 


Job  Position  Status  and  Organization  Type : 

The  customer  profile  in  this  survey  represents  five  (5) 
organizational  groups,  and  seven  (7)  job  positions,  the 
distribution  of  which  is  shown  in  Figures  5.1  and  5.2.  Aspects 
of  the  FY  2001  and  FY  2002  CS  Surveys  showed  similar 
organizational  status  percentages,  indicating  that  the  majority 
of  respondents  came  from  the  DOD  community  (48  percent) . 

However,  a  majority  of  FY  2002  CS  Survey  respondents  classified 
their  job  position  as  Librarian  (25  percent)  in  comparison  to 
last  year's  majority  of  Researcher/Analyst/Program  Manager  (18 
percent) .  Overall,  these  findings  continue  to  reflect  no 
significant  changes  to  job  position  from  previous  surveyed  years. 
It  is  interesting  to  note  that  of  the  respondents  who  classified 
themselves  as  DOD  users,  the  majority  chose  the  title  of  Engineer 
(22  percent)  to  describe  their  job  position. 


User  Organization  Type 


Organization 

FY  99  Customer 
Satisfaction  Survey 

FY  2001  Customer 
Satisfaction  Survey 

FY  2002  Customer 
Satisfaction  Survey 

DOD-Military  Services 
Agencies 

46% 

52% 

48% 

Gov't.  Contract/Industry 

40% 

35% 

38% 

Non-DOD  Federal  Gov't. 

6% 

1% 

5% 

Academia:  College/Univ. 

6% 

11% 

3% 

HBCU/MI 

N/A 

N/A 

3% 

Fig  5.1 

Job  Position 


Job  Position 

FY2001  Customer 
Satisfaction  Survey 

FY  2002  Customer 
Satisfaction  Survey 

Librarian 

16% 

25% 

Engineer 

16% 

17% 

Researcher/Analyst/Program 

Manager 

18% 

16% 

Technical  Info  Specialist 

8% 

9% 

Scientist 

14% 

9% 

Contracting  Officer 

2% 

5% 

Resource  Management/Financial/ 
Budget  Analyst 

4% 

4% 

Security  Officer 

8% 

N/A 

Other 

14% 

15% 

Fig  5.2 
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Intermediary  vs .  End  Users : 


The  two  categories  listed  in  the  chart  below  reflect  areas  on  how 
our  users  acquire  scientific  and  technical  information.  Overall, 
the  FY  2002  CS  Survey  revealed  that  65  percent  of  users  primarily 
act  as  information  providers  to  others  as  illustrated  in  Fig  5.3. 
Aspects  of  the  FY  2001-2002  data  on  intermediary  and  end  user 
usage  are  relatively  flat.  In  the  course  of  analysis  it  is 
important  to  note  that  the  majority  of  the  DOD  respondents 
overwhelmingly  reported  themselves  as  end  users  (70  percent) . 

This  could  be  due  in  part  to  the  majority  of  the  DOD  community 
responding  to  this  survey  coming  from  the  Engineering  field  (60 
percent ) . 
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Intermediary  vs .  End  User 


100' 

80' 

60' 

40' 

20' 

ZJ= 

0 

FY  99  Customer 

FY  2001  Customer 

FY  2002  Customer 

Satisfaction 

Satisfaction 

Satisfaction 

^Intermediary  Users 

83 

64 

65 

DEnd  Users 

16 

32 

35 

Fig  5.3 


Time  as  a  Registered  User: 

More  than  a  third  of  DTIC  users  have  been  registered  users  for 
six  or  more  years  (33  percent) .  This  was  a  profound  change  in 
comparison  with  last  year's  results,  in  which  the  majority  of 
users  responding  were  new  users  6  months  or  less  (47  percent) . 
The  data  displayed  in  Fig  5.4  below  defines  respondent's  time  as 
a  registered  user  with  DTIC: 


Time  Data 

FY2001  Customer 
Satisfaction 
Survey 

FY  2002  Customer 
Satisfaction  Survey 

6  months  or  less 

47% 

* 

Less  than  1  year 

* 

17% 

7  months-  2  years 

22% 

* 

1-2  years 

* 

29% 

3-5  years 

18% 

21% 

6  or  more  years 

13% 

33% 

‘Note-different  measurement  scales  (time  increments)  were  used  for  the  two  surveys 

Fig  5.4 
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PART  VI:  Communications,  Access  and  Information 

Requirements 


Information  Needs : 

The  vast  majority  of  users  (82  percent)  reported  a  high  level  of 
satisfaction  with  their  information  needs  being  met  by  searching 
DTIC's  collections.  However,  they  do  express  continual  need  for 
electronic  documents.  Respondents  also  expressed  concerns  for 
currency  and  future  informational  needs  to  be  included  in  DTIC's 
collections.  For  example: 

■  Chemical  and  Biological  Warfare  Information 

■  Homeland  Defense 

■  Security  Information 

■  Environmental  Cleanup 

Note:  These  four  subject  areas  (information  needs)  identified 

above  were  the  result  of  respondent' s  most  frequently  mentioned 
write-ins.  Also  FY  1999  and  FY  2001  CS  Surveys  did  not  ask  this 
question.  Therefore,  no  comparisons  can  be  made. 

Communication  Preferences  and  Satisfaction: 

This  year's  CS  survey  finds  the  Internet/Online  as  the  most  often 
used  communication  channel  to  contact  DTIC  (39  percent) .  It  is 
interesting  to  note  that  last  year,  the  telephone  was  the 
preferred  method  for  contacting  DTIC.  There  was  also  a 
significant  increase  in  email  and  internet  contacts.  Although 
the  FY  2002  CS  data  shows  a  slight  decline  in  telephone  contact 
(14  percent),  the  overall  preferred  communication  data  shows  an 
upward  trend  in  electronic  communication. 
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In  addition,  when  users  were  asked  how  they  would  most  like  to  be 
informed  about  DTIC  products/services,  updates,  etc.,  users 
responded  as  follows: 

■  69  percent  preferred  Email 

■  13  percent  preferred  Electronic  Mailing  List  (listservs) 

■  11  percent  preferred  Web  Site 

■  6  percent  preferred  Mail 


Comparative  Analysis  on  Effectiveness  of  Corporate  Communication 


FY  99  Customer 

Strongly  Agree  or 

Strongly  Disagree  or 

No  Opinion 

Satisfaction  Survey 

Agree 

Disagree 

New  Product  /Service  Changes 

84% 

3% 

13% 

Product  /Service  Changes 

74% 

8% 

18% 

Training  Opportunities 

59% 

10% 

31% 

Submitting  Documents 

51% 

8% 

41% 

Ordering  Documents 

84% 

4% 

12% 

Method  of  Payments 

69% 

6% 

25% 

Registration 

79% 

3% 

18% 

Customer  Support 

81% 

4% 

15% 

Annual  Conference 

68% 

2% 

30% 

Overall  Rating 

72% 

5% 

23% 

FY  2001  Customer 

Strongly  Agree  or 

Strongly  Disagree  or 

No  Opinion 

Satisfaction  Survey 

Agree 

Disagree 

Product  /Service  Changes 

81% 

4% 

15% 

Training  Opportunities 

63% 

5% 

32% 

Submitting  Documents 

46% 

9% 

45% 

Method  of  Payments 

56% 

5% 

39% 

Registration 

74% 

5% 

21% 

Online  Service 

70% 

6% 

23% 

Order  Status 

40% 

14% 

46% 

Annual  Conference 

57% 

2% 

41% 

Overall  Rating 

61% 

6% 

33% 

FY  2002  Customer 

Strongly  Agree  or 

Strongly  Disagree  or 

No  Opinion 

Satisfaction  Survey 

Agree 

Disagree 

New  Product  /Service  Changes 

83% 

16% 

0.7% 

Product/Service  Changes 

82% 

16% 

2% 

Submitting  Documents 

59% 

37% 

4% 

Training  Opportunities 

71% 

27% 

3% 

Ordering  Documents 

77% 

20% 

3% 

System  Disruptions/Downtime 

76% 

22% 

3% 

Overall  Rating 

75% 

23% 

3% 

Note:  Inquiries  on  the  specifics  above  varied  year  to  year 


Fig  6.3 
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